An empirical study of two typical locality preserving linear discriminant analysis methods

discriminant analysis methods Lishan Qiao, Limei Zhang, Songcan Chen 1 Department of Computer Science and Engineering, Nanjing University of Aeronautics & Astronautics, 210016, Nanjing, P.R. China 2 Department of Mathematics Science, Liaocheng University, 252000, Liaocheng, P.R. China Abstract: Laplacian Linear Discriminant Analysis (LapLDA) and Semi-supervised Discriminant Analysis (SDA) are two recently proposed LDA methods. They are developed independently with the aim to improve LDA by introducing a locality preserving regularization term, and have been shown their effectiveness experimentally on some benchmark datasets. However, both algorithms ignored comparison with much simpler methods such as Regularized Discriminant Analysis (RDA). In this paper, we make an empirical and supplemental study on LapLDA and SDA, and get somewhat counterintuitive results: 1) Although LapLDA can generally improve the classical LDA via resorting to a complex regularization term, it does not outperform RDA which is only based on the simplest Tikhonov regularizer; 2) To reevaluate the performance of SDA, we develop purposely a new and much simpler semi-supervised algorithm called Globality Preserving Discriminant Analysis (GPDA) and make a comparison with SDA. Surprisingly, we find that GPDA tends to achieve better performance. These two points drive us to reconsider whether one should use or how to use locality preserving strategy in practice. Finally, we discuss the reasons which lead to the possible failure of the locality preserving criterion and provide alternative strategies and suggestions to address these problems.